The Second Law of Thermodynamics and Quantum Feedback Control: 
Maxwell's Demon with Weak Measurements 
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Recently Sagawa and Ueda [Phys. Rev. Lett. 100, 080403 (2008)] derived a bound on the work 
that can be extracted from a quantum system with the use of feedback control. They left open the 
question of whether this bound could be achieved for every measurement that could be made by 
the controller. We show that it can, and that this follows straightforwardly from recent work on 
Maxwell's demon by Alicki et al. [Open Syst. Inform. Dynam. 11, 205 (2004)], for both discrete 
and continuous feedback control. Our analysis also shows that bare, efficient measurements always 
do non-negative work on a system in equilibrium, but do not add heat. 
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The amount of work that can be extracted from a ther- 
modynamical system, when it undergoes a process taking 
it from an initial equilibrium state Si at temperature T\ , 
to a final equilibrium state 52 at temperature T 2 , is given 
by the difference in the (Helmholtz) free energy, F, be- 
tween these states, where 



F = E-TS, 



(1) 



with E, T and S being, respectively, the average inter- 
nal energy, temperature and entropy of the system. For 
quantum systems the entropy is the von Neumann en- 
tropy, S = — Tr[plnp], where p is the density matrix of 
the state. The simplest example of this is the work ob- 
tained by the (quasi-equilibrium) expansion of a gas at a 
fixed temperature (isothermal). In this case the internal 
energy of the gas remains constant, and the work done 
by the gas is (Si - S 2 )/T = F x - F 2 = AF [1]. 

This relation between maximum work and free energy 
is true for the traditional thermodynamic processes - 
that is, ones that do not have access to the microstatcs 
of the system. If we measure the system, so as to ob- 
tain information about the underlying microstate, and 
perform actions based on this information, then we can 
extract more work. This is a process of feedback con- 
trol [2]. Of course, in this situation, a feedback controller 
is merely a Maxwell demon [3-11]. Sagawa and Ueda re- 
cently showed that the amount of extra work that can 
be extracted by a feedback controller, over and above 
the free-energy difference, is bounded by a measure of 
the information extracted by the controller [12]. In their 
analysis this bound was only achieved for certain special 
classes of measurements (e.g. von Nuemann measure- 
ments). Here we show, by generalizing the protocol of 
Alicki et al. [9] , that feedback controllers can always sat- 
urate the bound, irrespective of the measurement they 
make. 

The following analysis is divided into three parts. The 
first part introduces some important thermodynamic def- 
initions. In the second part we derive the relationship 
between free energy and work extraction with feedback 
control, building upon previous results by Alicki et al. [9] 
on Maxwell's demon. Lastly, we treat the feedback con- 



troller as purely quantum mechanical, eliminating the use 
of quantum measurement theory, to show that the sec- 
ond law is preserved by the control process, in agreement 
with Landauer's erasure principle [4, 13-17]. 

Quantum Mechanics, Work, and Heat: The average 
energy of a quantum system is given by E = Tr[Hp], 
where H is the Hamiltonian. We can therefore write 
dE = Tv[dHp] + Tr[Hdp\. In the past the work done 
on the system has been equated with the first term, 
dW = Tr[dHp], and the heat entering the system with 
the second: dQ = Tr[Hdp] [9, 10]. These identifications 
are not subtle enough for our analysis here, however, be- 
cause of transformations induced by the measurement. 
This necessitates splitting dp into a number of parts. 
Transformations that change the eigenbasis of p, such 
as unitary operations, and preserve the populations of 
the eigenstates (and thus the entropy of the system) cor- 
respond to work done on or by the system. Alterna- 
tively, processes that leave the eigenbasis of p fixed, but 
change the populations correspond to adding or subtract- 
ing heat, or information extraction. 

To examine the work done on a system by a quan- 
tum measurement, note first that the transformation of 
p caused by any efficient measurement can be written as 



ApA^/Tr^Ap], 



(2) 



for some operator A. The polar decomposition theorem 
allows us to write A = UP, where U is unitary, and P is 
positive, and this allows us to break up the transforma- 
tion into a unitary part (work done) and an information 
extraction part that reduces, on average, the entropy. 
The operator U can also include a unitary feedback oper- 
ation based upon the measurement result, and, therefore, 
can also be undone by the use of feedback. We will re- 
fer to measurements that have no unitary part as "bare" 
measurements [18]. 

To fully isolate the work done by the measurement we 
must consider the action of a positive operator P in a 
little more detail. If P commutes with the density ma- 
trix, the only change is to the entropy. But if P does not 
commute with p, then the action of the measurement 
changes both the eigenbasis of the density matrix, doing 
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work, and generates the entropy change (extracting in- 
formation). We will show in the analysis below that the 
average work done on a system in thermal equilibrium 
by a bare measurement is always non-negative. 

The thermal equilibrium state of a system at temper- 
ature T is 

e -H/(kT) 

PT = Tr [e~H/(kT)^ ' ( 3 ) 

where H is the system Hamiltonian. This state cap- 
tures the fundamental assumption of statistical mechan- 
ics (that all accessible microstates are equally likely). 
Given this assumption, it tells us how the entropy will 
change with other quantities in a quasi-equilibrium pro- 
cess. 

Extracting Work with Feedback Control: To obtain the 
amount of work that can be extracted by a feedback con- 
troller, we will start our system in an equilibrium state 
Pt, denoting the initial average energy by E and the ini- 
tial entropy by S. The first action of the controller is 
to measure the state, using an arbitrary quantum mea- 
surement described by a set of operators {P n }, satisfying 
J2 n = The operators P n are all positive, since the 
unitary operator associated with each measurement out- 
come will be determined by the feedback chosen by the 
controller. After the measurement result, which occurs 
with probablity p n ~ Tr [P%p], the state is 

Pn = PnPPn/Pn- (4) 

This is no-longer an equilibrium state, but its entropy 
and average energy are well-defined (its temperature is 
not). Call its entropy S n and its average energy E n . 
Note that in general E n ^ E, because the energy will 
have been changed by the measurement; this is the work 
done on the system by the measurement process. We will 
return to this later. 

Now comes the first part of the feedback control pro- 
cess. The controller performs work (reversible opera- 
tions) on the system to transform p n to an equilibrium 
state at temperature T. This is achieved by i) per- 
forming a unitary operation to transform the eigenba- 
sis of p n to the energy eigenbasis; ii) re-ordering the 
populations of the energy states so that these popula- 
tions decrease monotonically with increasing energy We 
will denote the populations (the eigenvalues of p n ) as 
X n j, and the corresponding energy levels of the system 
as e n j] iii) adjusting the Hamiltonian so that the sep- 
arations between adjacent energy levels are such as to 
set V n j{T) = exp(-e nj /(kT))/Z = \ nj , where Z = 
exp(~e„j/(fcT)) is the partition function, and we 
have defined V n j{T) as the populations of the energy 
eigenstates required for the system to be in thermal equi- 
librium at temperature T; iv) adjusting the Hamiltonian 
to produce an overall energy shift of the levels so as to 
return the average value of the energy to the initial value, 
E. This leaves the system in thermal equilibrium at tem- 
perature T, since the values of the populations, P n j(T), 



required for this to be true remain unchanged by the en- 
ergy shift. 

The above feedback extracts net work from the system 
of AE n = E n — E, preserves the entropy of the state, S n , 
and leaves the state in equilibrium at temperature T. 

In the second part of the feedback process, the con- 
troller performs an isothermal expansion of the system 
(decreasing the separation of the energy levels at fixed 
temperature), so as to return the entropy to the ini- 
tial value, S. This brings the system precisely back to 
its initial thermal state, since the energy, temperature, 
and entropy of the system have all returned to their ini- 
tial values. The isothermal expansion extracts AVT„ = 
T(S — S n ) of work from the system. The total work ex- 
tracted by the feedback controller in this cycle, given the 
measurement result n, is thus AW n = T(S — S n ) + AE n . 
Of course, the important quantity is the total average 
work extracted by the feedback, where the average is over 
the possible measurement results. This is, 

AW = T (s-J2PnSn) + ^> n A£„. (5) 

\ n / n 

We now examine the second contribution, the work ex- 
tracted deterministically by the feedback, ^2 n p n AE n . 
This is simply the average increase in the energy of 
the system caused by the measurement, Ai? moas = 
J2 n Pn{E n —E), being extracted back by the controller. If 
the measurement is classical, so that all the measurement 
operators, P n , commute with the initial state px (that is, 
the controller measures the systems energy) , then the av- 
erage density matrix after the measurement is the same 
as the initial state: p a fter = ^2 n VnPn = Pt- From this it 
follows immediately that AE'mcas = 0. Thus, as expected 
from our previous discussion of work and energy, when 
the measurement does not change the eigenbasis of the 
density matrix, then it does not, on average, add energy 
to the system. When the measurement operators do not 
commute with pt, then one has S^/Oafter) > S{pt), a re- 
sult shown by Ando [19]. Because the equilibrium state, 
Pt, is the state with the maximum entropy given a fixed 
value of the average energy, it follows that ~E a fter > E. 
We therefore have 

AE mcaB = i? a ftcr — E > 0. (6) 

Because the controller can always extract back as work 
all the energy added to the system by the measurement 
in a closed cycle (Eq.(5)), to preserve the second law 
of thermodynamics we must interpret A-E mea s as work 
added to the system, not heat. This is consistent with 
our observation that the action of a positive measurement 
operator induces a transformation of the density matrix 
eigenbasis. 

The total work extracted by the controller in a single 
cycle is the work extracted by the feedback process, mi- 
nus the work done on the system by the measurement, 
and is therefore AWo, = T(S - Y^nPn^n)- This is for 
a cycle in which the system starts in equilibrium with a 
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given free energy, and returns back to its initial state. It 
now follows immediately that the work extractable by a 
feedback controller when starting in state <Si with free 
energy F\, and ending in state 1S2 with free energy F2, is 

AW th = &F + t(^S-J2 PnS,)j , (7) 

where AF = F1—F2. The right hand side of this equation 
is the upper bound derived in [12]. 

The quantity A5 moas = S — ^2 n p n S n , being the aver- 
age entropy reduction provided by the measurement, is 
always non-negative for efficient measurements, a result 
due to Ozawa [20-22] . This is a key quantity in quantum 
feedback control even outside thermodynamical consid- 
erations [23], and reduces to the classical mutual infor- 
mation when the measurement is classical. 

The Second Law: The feedback control process we 
have just described reduces the entropy of the bath, on 
average, by A^meas during the isothermal expansion of 
the system (the system gains this amount of entropy from 
the bath). Since the final entropy of the system is the 
same as its initial entropy, the whole process will break 
the second law of thermodynamics (reduce the entropy 
of the universe) , if the entropy of the controller does not 
increase by at least AS mcas . The simplest way to show 
that the entropy of the controller does increase by the 
required amount is to treat the controller fully quantum 
mechanically. This allows us to treat the whole feedback 
process without using quantum measurement theory. As 
pointed out by Wiseman, any feedback control process 
based on explicit measurements (that is, with a controller 
whose states are classically distinguishable, and thus do 
not exist in superposition states) can always be imple- 
mented with a quantum controller, without any explicit 
measurements [24]. 

We will denote the controller as C, and the system as 
S. The measurement process is completely described by 
a unitary operation acting on the space of both systems. 
The controller has N states, \n), n = 0, . . . , N — 1, where 
N is the number of measurement results. The initial 
state of the controller is |0), and that of the system is, 
of course, pT- A joint unitary operation correlates the 
systems so that the joint state becomes 

Pes = y^p n ln)(n| ® p n + 2J \n)(m\ (g> a nm . (8) 

That this is possible is guaranteed by the fact that the p n 
are given by Eq.(4) [25]. The a nm are matrices with the 
same dimension as the p n , but we will not require any 
further details about them. A second joint unitary opera- 
tion now performs feedback, applying a different unitary 
transformation to S depending on the state of the con- 
troller (each state, |n), of the controller is the equivalent 
of measurement result n in our previous analysis). This 
unitary has the form 

U{ h = Y / \ n )( n \® U n, (9) 
n 



where U n acts on the system. These unitarics perform 
the reordering of the eigenvalues of p ni and the change in 
the system Hamiltonian (the energy levels) to bring the 
system into a thermal equilibrium state and adjust the 
average energy. 

The controller then performs the final part of the feed- 
back in which it expands S isothermally to extract the 
work. This cannot be described purely as a unitary op- 
eration, because it leaves the bath in a state of different 
entropy for each value of n. Because these different states 
of the bath are necessarily macroscopically distinct (they 
have different entropy, and are therefore macroscopically 
distinguishable), this fully decoheres the controller in the 
basis |n). This can be described using a unitary of the 
form given in Eq.(9) that maximally entangles the con- 
trollers basis states, |n), with an auxiliary system of the 
same size, followed by tracing over the auxiliary system. 

Now, the result of the feedback operation on each state 
p n is to transform it to a final state p„ nal with entropy S, 
temperature T and average energy E. Since the temper- 
ature and entropy of all the p^ nal are the same, they have 
the same set of eigenvalues (the same distribution of pop- 
ulations). Since the average energy is also the same for 
all these states, they must also have the same set of en- 
ergy levels. Thus for every value of n (for each state \n) of 
the controller) the system has the same final Hamiltonian 
and the same final state, px- Because of this the state 
of the system and controller factor, and we can write the 
final joint state as p^ al ® Pt- Because the system is at 
thermal equilibrium, the joint state of the system and 
bath also factors. However, the state of the controller 
and the bath does not factor - this is because, in general, 
the bath transfers a different amount of entropy to the 
system for each value of n, and is thus left in a different 
state for each value of n (as discussed above). Since the 
probability is p n that the state of the controller is \n), 
and since the different states of the bath are classically 
distinguishable, the final state of the three systems is 



Pfinal = 




If we denote the initial entropy of the bath as 5b, the 
entropy of each final bath state p^ ath is Sb — (S — S„). 
The total entropy of the final state is therefore 

■SVfinal] = S({p n }) + ^PnS n + S B , (H) 

n 

where S({p n }) = — J2 n Pn lnp n is the entropy of the dis- 
tribution of measurement results. Since the total initial 
entropy of all three systems is S + 5b, the total change 
in the entropy of the universe for the cycle is 

AS** = S({ Pn }) - AS meas . (12) 

The second law then follows from Nielsen's result [21], 
which states that for every measurement, S({p n }) is an 
upper bound on ASVeas, and thus AStot — 0- 
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For the controller to start the cycle again, it must re- 
turn to the state [0). To do this it simply connects itself 
to a fourth system with dimension TV in a fixed state, 
|0), performs a (unitary) swap operation between itself 
and this fourth system, and then dumps the fourth sys- 
tem into the thermal bath. This leaves the controller in 
state |0) with zero entropy, and increases the entropy of 
the bath by S({p n }). The feedback control cycle is now 
complete: work AW = TAS meas has been extracted, the 
controller and system are back in their initial states, and 
the entropy of the bath has increased by A5 t ot- 

We note that the feedback control cycle is only thermo- 
dynamically efficient (preserves the entropy of the uni- 
verse on average) when S({p n }) = AS'meas- This is 
only true if the measurement operators P n commute with 
Pt [21], so that the measurement is classical. This means 
that the feedback controller only preserves the entropy of 
the universe when it makes measurements of energy. 

We have so far only explicitly considered feedback con- 
trol with efficient measurements. An inefficient measure- 
ment is one in which the controller makes an efficient 
measurement, but throws away some information about 
the measurement result [25]. All inefficient measure- 
ments can be described by the set of operators A n j , where 



12njAnjA n j = I, and as before n labels the measure- 
ment results. The final state of the system given result n 
IS Pn = Y*j A n]PTA\j/p n , with p n = YA Tr[A nj A nj p T \. 
With these new definitions of p n and p ni the above analy- 
sis of the feedback cycle goes through unchanged, except 
that S n is not necessarily less than S. In this case, the 
ability of the controller to extract work from the sys- 
tem can be reduced by the measurement, rather than in- 
creased. Because of this, inefficient measurements can 
add heat to a system, as well as doing work. 

Lastly, we note that we have performed all our analysis 
with feedback from a "single-shot" measurement. This 
is usually referred to as "discrete" feedback control, to 
distinguish it from feedback control that uses continuous 
measurement [26]. However, the analysis we have pre- 
sented can be easily modified to derive the same result 
for continuous feedback control. All we have to do is 
observe that each step in the feedback cycle can be per- 
formed infinitesimally. (A single infinitesimal time-step 
of a continuous measurement is described by a measure- 
ment in which all the operators, A„, arc infinitesimally 
close to the identity [26].). Our results above thus apply 
to all feedback control, whether discrete or continuous. 
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